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SUMMARY 


This paper describes the methodology developed for a defense 
department laboratory for project selection and the results of 
use of the computerized ranking procedure for other complex 
Managerial decisions. In addition to project selection, the 
system is feasible and instructive in personnel selection, 
comparison of management strategies, purchasing decisions and 
merit-based distribution awards. The massive input required by 
Many existing systems is not necessary for this absolute 
deviation ranking system. Up to 20 attributes and 200 projects 
can be compared. Measures of consistency and reliability are 
Proposed for estimating the validity of the final weights 
considering missing values. Preliminary experiments and actual 
use of the system provide empirical validation and show the 
results are satisfactory. Further research is underway for the 
refinement of the system. The impact on reliability of the 


number of items compared, the comparison scale employed and the 


number of missing observations provide fertile fields for future 


work. Extensions of the basic and additive individualized 
Multicriteria models provide for voting, hierarchy and 
constraints options for group decision making. The current 
version of the system is an organized, structured way of making 
decisions which enhances the identification and subsequent 
refinement of criteria considered, and provides a concise, highly 
credible format for presentation and justification of decisions. 
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SECTION I 


INTRODUCTION 


Many organizations and individuals experience difficulties 
when considering which projects to give their support. Other 
complex managerial decisions creating problems include, for 
example, 1) developing policies for ds oucion of personnel 
incorporating nondiscriminatory and aftiimative action 
procedures; 2) comparison of management strategies such as 
running a university on the semester or quarter system; 3) 
purchase of equipment like compatible, user-friendly computer 
hardware and software; and 4) distribution of awards, funding, 
promotions and/or tenure based on merit. Crucial to the 
managerial decision making process is the ability to communicate 
decisions to subordinates and superiors. What is needed is an 
organized, structured way of making decisions which enhances the 
identification and subsequent refinement of criteria considered, 
and provides a concise, highly credible format for presentation 
and justification of decisions. | 

The task addressed in recent research by the authors was to 
extend existing methodologies in the design and implementation of 
a Simple but effective computerized ranking procedure for a 
defense department iaboraccey: The immediate goal was to provide 
a documented procedure for selection of projects to be funded. 
Ongoing research includes further refinements and training of 


personnel in the use of the system. Other research in progress 
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by the authors involves the application of the ranking system to 
the four areas listed above and a modification of the program to 


allow implementation on personal computers. 


SECTION II 


DISCUSSION OF EXISTING RANKING METHODS 


Numerous efforts have been made to apaueae a workable 
methodology for project selection for the defense department, but 
past efforts have failed [1]. Further attempts to develop 
utility type models [2] have met only ‘limited success because 
consistency must be assumed and the process of using the models 
is very time consuming. An extensive literature survey was made 


in earlier work by the investigator [3]. 


Ls THE VON NEUMANN-MORGENSTERN AXIOMS 
; The basis for most modern utility research is Theory of Games 
' and Economic Behavior [4] in which the use of expected utility 


hypothesis is explained. This approach is based on the idea that 
cS a utility can be assigned to each outcome, developed in the. 
rs 1940's by von Neumann and Morgenstern. Since they are basic to 


all decision theory, the axioms and notation are included here. 


a. Notation 
1. ©“ = y indicates that the decision maker is indifferent 
between the outcomes x and y. 
2. x > y indicates that the decision maker prefers the 


Outcome x to the outcome y. 


2 


een, 


= 


3. x < y indicates that the decision maker prefers the 
outcome y to the outcome x. 
The Axioms 
For two consequences x and y, one and one of the 


following relations holds: 


This axiom provides for comparability of consequences. 


If x > y and y > z, then x > z. , This axiom establishes 
transitivity. 

x < y implies x < px + (l-p)y, where p is the probability 
of consequence x(0 < p < 1). This means that if y is 
preferred to x, then the decision maker will prefer a 
lottery between x and y to the certain outcome x as long 

as the probability of receiving y in the lottery exceeds 
Zero. 

x > y implies x > px + (l-p)y. This is the converse of 
axiom B.1l. 

x < z < y implies: There is a p (0 < p < 1) Such that the 
decision maker has the following preference: px + (l=p)y<z. 
This axiom establishes the commensurability of the 
consequences. That is, no matter how superior outcome y is 
perceived to be with respect to z, there is a sufficiently 
large probability p such that the certain outcome z is 
preferred to the lottery over x and y. 
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B.4. %x* > z > y implies: There is a p (0 < p < 1) such that 
px + (l-p)y > z. This is the converse of axiom B.3. 

cC.l. px + (l=-p)y = (l-p)y + px. This is the utility theory 
analog of the mathematics associative law. 

C.2. plqx + (l=-q)y) + (l-p)y = pqx + (l-pq)y. This is the 
utility theory analog of the mathematics distributive law, 


where q is also a probability (0 <q <1). 


Ze STATISTICAL DECISION THEORY 

The next major contribution was made by Savage in 1954 [5] 
in his The Foundation of Statistics. ‘Axioms incorporating 
subjective expected utility were estimated by the decision maker 
during the décision making process. 

By applying statistical theory to actual problems, Bayesian 
Or statistical decision theory was developed by such men as 
Schlaifer [6], Raiffa and Schlaifer [7], and Pratt, et al. [8]. 

The next step was the investigation of specific charac- 
teristics of decision problems. Methods to assess utility 
functions were proposed by Pratt [9], Schlaifer [6], and Novick 
and Lindley [10]. The assignment of subjective probabilities was 
suggested by Winkler [ll], Edwards [12], Schlaifer [13], and 
Tversky and Kahneman [14]. Group decision problems were studied 
by Arrow [15] and Keeney and Kirkwood {16], and the question of 
multi-attribute preferences was discussed by Fishburn [17,18,19], 
Pollak [20], Raiffa [21], Keeney [22,23], and Keeney and 


Raiffa C2) 
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3. MULTI~ATTRIBUTE UTILITY THEORY 

Until now, only one attribute had been considered. A method 
was developed by Keeney and Raiffa [2] for dealing with more 
than one. They distinguished between a value function, a multi- 
attribute preference function where each decision outcome can be 
measured with certainty for each attribute, and a utility 
function, a multi-attribute preference function with at least one 
ry evaluation uncertainty. 


The simplest utility function for n attributes is 


n ¢ 
u(x) = I klul(x.), (1) 
itl llsi 


where x = n=vector of attribute value, Kir Kor seen Ke 


u(x) = utility of the vector x, 
ki = weighting of attribute Kin i=1,2,...,n, 
x5 = a value of the attribute Xie 
uj (x) = marginal utility of Xie 


Fishburn [18,19,24,25,26] noted that the additive utility 
model for two attributes x and y is applicable if and only if x 
and y are additive independent. Thus, the paired preference 
comparison of any two lotteries depends only on these marginal 
probability: distributions. He also extended the two-attribute 
independence to the general n-attribute case and developed the 


multilinear utility function later refined by Farquhar [27]. 
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n n n 
u (x) = £& k,u,(x.) + £ » k, .u.(x, Ju,(x.) 
i=l 11 z i=l j>1 1 i J 3 


n n n 
+ z Zk... u, (x, du.(x. du (x ) +... 
feb jotomg tI 4 Dd me 


+ kK) 23..0n U(x) ug(xo).-eu (x) (2) ? 


where the notation is identical to that for (1) except 
for the additional weighting constants (the k's) with 


multiple subscripts. 


¢ 
Another important concept is utility independence. A set of 


wee a QR ee es IS ee eR a > 


attributes is said to be jointly utility independent if and only 
if each of the attributes is utility independent of its 
complement (the remaining attributes). The multilinear utility 
function is applicable only when the attributes are jointly 
utility independent. 

If the attributes (Xp rXoreee eX) are mutually utility 
independent, then the utility function u(x) takes the following 
multiplicative form: | | 

_ n 
ku(x) + 1 = = (kk us (x, ) + 1), (3) 


where 


l. u(x) is normalized by TG Sas ree ) 


* 


* 
1 Xo reas eee ) 


and u(x 
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2. u, tx) is a marginal (conventional) utility function 


* 
on X, normalized by u.(x,°) = 0 and u.(x, ) =1, 
i aa | » aaa 
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to 
3. ks = u(x, eK: V4 


4. k is a scaling constant determined by an iterative 


a 


procedure. 


Keeney [28] shows that when the sum of the n scaling 
constants (the k,"s) is equal to one, (3) reduces to the 
additive model (1). : 

While the multiplicative model is easy to assess, (2°53) 
subsets must be verified for mutual utility independence, 


Fortunately, this can be simplified by using these 


equations, proven equivalent by Keeney and Raiffa [2]: 


1. Attributes Xie Xoe coor x are mutually utility 


independent. 
Des x; is utility independent, i=1,2,...,n. 


re 3. (X.,- ) is utility independent for i=1,2,...,n 


x eee x 
i i+l’ ae 9 


on ‘ ee 
. and (Ky eX ores eX) is utility independent. 


s at pom 
‘ 


4. XoXo 4y) is utility ‘independent for i+l,2,...,n-l, n> 3. 


5. Xx) is utility independent and (Xe me) is preferentially 


independent for i=2,3,...,n, n > 3. This condition can 


is ee ao a Aad 
ae Te ae 


be generalized to require that any single attribute, not 
necessarily X, is utility independent. 

In these equations, cases 2 through 5 require only n 
assumptions. In equation 5, the term preferential independence 
refers to value functions, or trade-offs under certainty. A pair 
of attributes X, and X. is preferentially independent of its 


1 2 


complement, Xj 9 if the conditional preferences over (X,, X.) 


pe <2 

given some value x, or Xx, do not depend on Xho: . 
This method has the additional advantage of having been 

successfully applied, most notably in the summer of 1971 by 

de Neufville and Keeney [29] in choosing the site for a new 

airport in Mexico City. 
One-way utility independence, developed by Keeney [23], is 

the decomposition of a two-attribute utility function u(x,y) when 


only one is utility independent of the other. 


If X is utility independent of Y, then 


x 


° 0, ulx py) - u(x?,y) 
u(x,y) = ulx,y) + ulx,y>) =e, (4) 


u(x Wo) 
fe) ro) x * : 
where u(x,y) is normalized by u(x’,y ) = 0 and u(x ,y ) = 1. 
An equivalent decomposition was derived by Bell [30] using 
conditional utility functions. 
If X is utility independent of Y, then 


* * 
u(x,y) = u(x?,y ) u(y/x°) + u(x ry) u(x/y>) 


* 
+ (l=-u(x ,y°)) uly/x ) u(x/y?) 


* 


- ulx?,y ) uly/x°) ulx/y>). (5) 
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If Y is utility independent of X, then (2) is applicable, 

u(x,y) differs from u(x/y) in that u(x,y) is the decision 
maker's utility for the joint outcomes x and y and u(x/y) is the 
decision maker's utility for the outcome x, given that the 
outcome y is received. 

Another example with a simple and efficient decomposition 
and a difficult to verify requisite is interpolation 

. independence, developed by Bell [30]: 
"If the conditional utility function u(x,y) may be defined 


by a relation of the form 


¢ 


wlx/y) = B (y) nix/y") 
+ (1-fly)) ulx/y?), (6) 
then I will say that X is interpolation independent of Y 
and write X(II)Y." 
If X and Y are jointly interpolation independent, then 


u(x,y) = au(x/y°) + bu(y/x°) = ku(x/y?) uly/x>) 


+ 


(k-a) ul(x/y°) uly/x*) + (k=b) u(x/") 
uly/x°) + (l-k) u(x/y*) uly/x*), (7) 


where a = u(x*,y°); b= u(x>,y*) and k is an independent 


constant. 

4. ANALYTIC HIERARCHY 

This method, developed by Saaty [31], relies on reducing 
systems to pairwise comparisons of components grouped by shared 
characteristics. These pairs are compared on some scale. 
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After all values have been entered, the maximum eigenvalue 


and its associated normalized eigenvector are calculated by 
computer. This eigenvector represents the best weighting for the 
factors. 


The consistency factor is 


eT ee ERE 


r - : 
CI = Consistence Index = ee 
n-1 
RI = Random Index = determined for n by experiment and . 


reported in Saaty [31] (p. 31), 
Consistency Ratio = CI/RI : 

If the consistency is greater than 1.00, then a preference 
of a<b<c<a may exist. If the consistency is greater than .1 and 
less than 1.0, then there may be a problem of degree. The 
decision maker may have said that a is strongly preferred to b, b 


is strongly preferred to c, and a is strongly preferred to c. In 


this case, a should be very strongly preferred to c. 


3% EVALUATION OF AVAILABLE METHODS 
The importance of allowing for multiattributed decision 
making is elaborated by Winkler [32]. A variety of approaches ° 
to developing multidimensional evaluations have been suggested 
in the literature. The decisions made during project selection 


and many other ranking processes are such that no ideal answer 


exists. Fuzzy goal programming and fuzzy multicriteria pro- 


gramming techniques are reviewed by Ignizio [33] and suggested for x 
a | 

n 

use in decision making. The subjective nature of decisions such ' 
10 . 
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as project selection lends itself to preference comparisons [34]. 
These comparisons can be consolidated into a ranked and weighted 
set of projects using Saaty's Analytic Hierarchy Process [31] or 
linear and mathematical programming methods [35,36]. Holistic 
judgements have been used for small sets of comparisons and 
empirical validation [37] does seem appropriate for determining 
the credibility of decisions based on personal preferences. 

Saaty [31] overcomes the consistency difficulty for small problems 
of fifteen or fewer options, but extensive input is still 
required. The Saaty procedure calls for a complete matrix of 
paired comparisons on some scale. This would amount to 

n(n-1)/2 comparisons, or 105 in a case with 15 options. A recent 
study by Zahedi [38] employed a Saaty~=based method of multiple~ 
criterion ranking, but the author's example (p. 103) was a 
totally consistent matrix which is not realistic in practice. 
Hannon [39] concurs that the consistent case is trivial since any 


column of the input matrix is the weight vector for the solution. 


Experience of the authors [3] has shown that in order to achieve 


consistency with a large matrix, the scale must be expanded much 
beyond Saaty's scale. Another study, by O'Grady and Menon [40], 
uses multiple criteria, but does not provide the flexibility for 
each user to choose his own set of criteria. Research by the 
authors shows the selection of such criteria is virtually 
unrepeatable. A recent article describes a preference weighting 
scheme using absolute deviations [41]. This article describes 
how models based on absolute deviations may be used to weight any 
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number of projects by either individuals, groups, or in 
hierarchies. However, the particular formulation used in [41] 
assumes the comparisons are in reference to an ideal. [In 
applications considered by this paper, no ideal is known, so a 
modified model is needed. 

In summary, some pertinent methodological problems remaining 
are 1) decreasing the amount of input needed ‘for single and 
multiattribute decision making, 2) allowing the comparison of 
large numbers of items and 3) providing an estimate of the 
consistency of the results. These problems are addressed, but 
not totally solved, by the methodology developed and implemented 


in this study. 


SECTION III 


METHODOLOGY 


1.  SAATY METHOD 

The procedure developed by Saaty [31] asks the decision 
maker to state his preference on some scale, which Saaty 
recommends be based on divisions of seven or nine. These 


preferences may be represented in a matrix, A as: 


2 be A eseid Joao n 


Ll. .6, wees (8) 
i 


where ae represents the degree of preference the decision maker 


has for project i over project j, and a = ne aaa 
Saaty then finds the maximum latent root, \. In the perfectly 


consistent case, A= n. 


If A= n, then each preference ate is exactly consistent and 
WwW . 
a.. pad (9) 
1) Ww 


Thus, multiplication of the consistent A matrix by the vector W = 


(woe Woeeew ) of weights gives W, as illustrated below: 
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Tah 


sf ae ua 


wr an bab Wy 
Ws Wh Ww 
(l=n) eee 
A (10) 
(l=n) w 
n 
“1 (Wo)teeet ab ee Wy 
= wo Wn w 
2 
é 
= =w (11) 
eee tw —n(w_) 
w 
n 
Inconsistencies in preferences stated by the decision maker 
result in a less than perfect weight vector and increase the 
difference between . and _n. Saaty is therefore able to measure 
consistency using oe = SS. He further refines the consistency 
n-l 


estimate for size using statistics determined from simulations. 
However, this technique lacks the ability to estimate missing 
values. 

2. ABSOLUTE DEVIATION PROCEDURE 

Using an absolute deviation procedure, the model may be 


formulated using the A matrix as 
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ute 


k + - 
: re is 
min ee (ey ey ) Zz (12) 
+ - 
: = ; =e Pe = 0 
ST ws are ee k k (13) 
n 
zr w, =l (14) 
i=l. ? 


where 
for ranking n projects, for each i and j from 1 ton, 
k is the number of comparisons, n-l < k < n (n-1)/2, 
we is the weight of project i, 
aij is the degree of preference for project i over 
project j for comparison k, 
€. and a. are the positive and negative errors for 


comparison k, and 


Z is the value of the objective function. 


k + - 
For the perfectly consistent case Z = f Ce re ) = 0 therefore 
k=1 
aig = =< as before and the wi for the absolute deviation must be 
wi 


exactly the same as using the normalized eigenvector. Of course, 
as the inconsistency increases, the W; obtained by the two 
methods will not be the same. 

A crude measure of consiStency may be obtained where there 
is redundancy by finding Z/(n-1) as illustrated in the following 


example. Further statistical work is now being conducted by the 
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author to adjust this consistency for the amount of redundancy 
and size of n. 
3. EXAMPLE OF BASIC ABSOLUTE DEVIATION METHOD 

A decision maker must choose among three alternatives. He 
decides that he can make quality statements about the paired 
relationships; hence he states he prefers two strongly over one, 2 
three strongly over one and three very strongly over two. 
Using a scale of one to nine, these statements are translated 


into the quantitative matrix. 


1 2 3 
1 ail ay2 a,3 1 5 5 
= 2 a5) a2 a53 = 1/5 1 7 (15) 
3 a3] ax a33 1/5 1/7 1 
This matrix is translated into the model given as follows: 
+ - + - + - 
Min e, €, +€, +E, +E, +é, =Z (16) 
+ _- 
: - -€ +€E = 0. 
ST: Wy OW, l 1 ; z 0. (17) 
7 Ey FE = 0° (18) 
Ww Tw 2 De, “ig ” 
wa - Sw -€, te, = 0+) (1:9:) 
+ + = 
wy Wo Ww, l (20) ; 
The solution of this problem is w) = 0.8140, Wo = 0.1630, W = 


0.9233, 2 = 0.6977, 
A measure of consistency might be formed by dividing Z by n-l, 


hence a crude measure for this ;roblem would be 0.35. Also note 


16 


om oo atta’ yee Sg, Bee ta,» ih wl oe nS etry, Gat San cide, eons 
: 2 . Egret see ee Sy ates * = i ie rege * . sf ae. ter ee op - - fos . : - ~ le ie Les Ce ea we a ~? . » 

Be a Mc Rent i cat eR lO eal sel eat eel yt ee ea “Bh Ee PRM Ga Man, tpn In BE PO in Hi a Te ees ae Ln ene hall a ee 
* = 2 . 


Be ee et my os Sel ees te be le a) epg Be ae ea 


Ree it tet) RN tat ea me ROR BS eA ye a wet re, we phage oe 


that the a.. implied by the final solution are a =i =~ = 5, 
1j 12 W2 7 
413 = : 35 and, a53 = 7. 


This suggests to the decision maker that perhaps he 
understated the relative importance of project three overall. 

The decision maker may wish to alter some choices. _The procedure 
indicates a likely place to look for errors ana will 
automatically estimate and insert missing values. 

4, EVALUATION OF ABSOLUTE DEVIATION METHODOLOGY 

Evaluation of the Saaty and absolute deviation techniques 
reveals that, for consistent comparisons, the two methods are 
identical, even when many comparisons are omitted in the absolute 
deviation technique input. As few as n-l interlinked comparisons 
will produce results equivalent to the Saaty results for n(n-1)/2 
comparisons. 

If the available comparisons are not linked, erroneous output 
is produced. For example, if there is a project or projects for 
which no comparisons are made, the solution will iets @etont 
of zero for these projects. 

Assuming that all alternatives are linked by comparisons, 
the robustness of the consistency estimate varies directly with 
the number of redundant conpaeiecdes The redundancies reduce the 
impact on the validity of the final weight vector of a single 


very wrong comparison. 
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The absolute deviation method thus offers the potential of 
being identical to the Saaty method in the consistent case. It 
addresses the weakness of the Saaty method, offering a relatively 
simple solution to the problem of estimating some missing values. 
An additional advantage of the absolute deviation method is that 
it can readily be programmed using standard L.P. software 
packages. 

In order to incorporate the strengths of the absolute : 
deviation technique into a managerial decision-making program 
under development, the basic absolute deviation model has been 
expanded to provide for multiple criteria, committee and/or 
hierarchical decision making and to allow constraints. 

3. ADDITIVE MODEL 

The additive model may be easily employed by first 
determining the attributes relating to given decision 
alternatives. The weights of the attributes may be obtained by 
use of the basic model. In practice an independent check of one 
Or more of the weights by another method, such as trade-offs, may 
be desirable. The investigator must next use the basic model on 
the alternatives for each attribute and combine the resulting 
project weight vectors into a matrix, F, which is the multiplied 
by the attribute weight vector, W, to give the final weight 


vector, Wes 


FW = W (21) 


6. VOTING MODEL 
The simple voting model may proceed by assuming each of the 
m members is equal and, for each member, scaling that member's 


sum of weights for the n protects to be 1/m. Thus, 
n 
cow. = 1/m (22) 


where we is the member's weight for project ie 
The group vote for project i will be the sum of the members' 
votes for project i. An alternative, through rarely feasible in 
practice, would be for each member to weigh the other voting 
members. The sum of project weights for member j would then be 
scaled to sum to ae instead of 1/m. This particular version 
might also be applicable to elections. 

A voting model, which might produce better results when 
there are small groups of voters with only a limited number of 
choices, is available (but not yet programmed or tested). 

After each of the V voters has ranked the same n projects 


and is satisfied with his individually produced weights Wiyt 


Wo +«+ewW _» the K_ comparisons, a, 
2v nv Vv 


. » Of each voter, v, are 
ijv 


combined to produce final weights w 


1’ WoreeW. using the model: 
V K + 7 
min = By, ter, ee )=2Z (23) 
vel k =1 vk ‘vk 
Vv v Vv 
+ ~- 
ST: Wigs - Bat Ma € k + € k = 0 (24) 
1 13 JV Vv Vv Vv y 
For VIleceVi ko = LeesK 2 1 < 4K mye 1X 4. < ony 
Vv Vv = = - 7 = 
nal. < kK. Soomntn= 1) 72 
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This a priori optimal model might be more desirable than the 
simpler additive voting model where each person's vote has. equal 
weight. | 
le HIERARCHY MODEL 
Finally, the most difficult problem arises in hierarchies, 
where it is desirable to weight or rank tasks at lower levels. 


For example, a division may have two departments, as in Figure l. 


DIVISION 1 


Figure 1. Hierarchical Organization. 
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Figure 2. Combined Preference Matrix. Submatrices A, and Ag are 
Derived from Group 1 and 2 Approved Weights. Division 
Chief Fills in Some Connecting Comparisons (*'S) in 
Shaded Submatrices. : 


Groups 1 and 2 can then prioritize within their respective 
organizations. The division leader may accept their ranking, 
discuss individual comparisons and reweight, or substitute his 
own order (he is in charge). After a satisfactory set of weights 
is obtained within each group, the division head must compare 
one or more (m - 1 or more in general) tasks from each group. 
That is, A, and A, in Figure 2 are partially or completely filled 
in. Optionally, additional projects may be added by the chief. 
Such an example is found in the User Guide, Appendix A. This 
problem is again solvable by Our procedure. 

8. CONSTRAINTS MODEL 

Resource allocations may’ be made using the final weights 


from any model. If budget and manpower are to be considered as 


separate constraints, the model is as follows: 
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(29) 


where Wi is the weight of project i, for bie the n projects, 

Xs must equal 0 or l, 

b. is the budget estimate for project i, 

B is the available budget, 

m is the manpower required for project i, and 

M is the available manpower. 
In the above model one can easily add further breakdowns, such as 
6.1; 6.2, or 6.3 program budgets, or skill level and grade 
breakouts for manpower. Other restrictions such as forced entry 
of micro-managed projects or assuring a minimum level of activity 
for each organization may be conveniently entered. 

9. COMPUTATIONAL METHODS 

The models shown have been programmed for the "VAX" and "IBM" 
computers. The basic code used was "LINDO" with a FORTRAN matrix 
generator and a report writer developed for interactive input and 
output. 

Tests of these models indicate that a full matrix of about 
55 projects can be weighted, and that 300 projects can be 


weighted if a minimally dense matrix is input. 


22 


ar ee Pee id See 
-* ° 


foxd . é 2 
toe. ae 


4 “gehen ate‘ ate mt, ob. “oi, * Et e +s) + q aye ea a q = ar 
ee we Ve - [Sa Ee I * — — a ate Ta. LL. bate oD em ! aS Ol ta hee Se Ed ci, 82: 


SECTION IV 


RESULTS 


The decision-making results from use of the models 
implemented as formulated above are illustrated by the following 


examples and summaries of output. 


l. SAATY VERSUS ABSOLUTE DEVIATION 

First, the Saaty and absolute deviation techniques are 
compared to establish the feasibility of analysis for large 
numbers of projects, n, and when increasing numbers of 
comparisons, k, are missing. The runs reported are illustrative, 
not a orecise quantification of the relationship between number 
of missing values and consistency. 

Empirical observation of trends in a realistic example of 
good but not perfect consistency show that the objective function, 
Z, decreases with the number of comparisons, k. (Omitting a 
comparison is equivalent to omitting a constraint.) As k varies 
from its maximum n(n-1)/2 to its minimum, n-l, Z approaches zero, 
but the final weights stay the same. Tec further substantiate 
this observation, two input matrices from Saaty [31] were tested 
as input to the absolute deviation model. Results are summarized 
in Tables 1, 2 and 3. Comparison of Saaty and absolute deviation 
techniques show the increase in consistency (decrease in Z) with 
fewer comparisons and the effect of decreasing consistency on the 


weight vector. More details are found in Appendix B. 
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TABLE 1 


TEST EXAMPLE 


IMPACT ON WEIGHT VECTOR AND OBJECTIVE FUNCTION OF 
DECREASING NUMBERS OF COMPARISONS WHEN 
FULL MATRIX SHOWED GOOD CONSISTENCY 


Absolute Deviation 


0.045455 


Wo 0.204545 
W3 0.545455 
Wy 0.068182 


0.136364 


Objective 2 0.318181 
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TABLE 2 
SAATY SCHOOL EXAMPLE 


IMPACT ON WEIGHT VECTOR AND OBJECTIVE 
FUNCTION OF DECREASING NUMBERS OF 
COMPARISONS WHEN FULL MATRIX 
SHOWED MODERATE CONSISTENCY 


Objective Z 
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TABLE 3 


SAATY SCHOOL EXAMPLE 


IMPACT ON WEIGHT VECTOR AND OBJECTIVE FUNCTION 
OF DECREASING NUMBERS OF COMPARISONS WHEN 
FULL MATRIX SHOWED POOR CONSISTENCY 


Absolute Deviation 


Objective Z 1.4034 
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These preliminary results illustrated the feasibility of the 
absolute deviation procedure for giving close estimates for the 
weight vector even when many comparisons are omitted if the 
comparisons input are consistent. The patterns observed in the 
relationship between the objective function, Z and the numbers of 
missing observations spurred additional study. 

2. ADDITIVE MODEL RESULTS 

An additive model was used to provide (optional) multi- 
criteria capabilities. The system was used in an actual hiring 
decision process where 7 applicants were ranked on 7 criteria. 
Thus an attribute matrix, A, was generated, and for each 
attribute, the applicants were rated with respect to that 
attribute. This produced 7 more matrices, B - H. To test the 
impact of missing values, for each matrix A - H, and for 
r=0,3,5,6,8,10,and 15, comparisons were randomly selected to be 
omitted. Three different random selections of r comparisons were 
made for each matrix and each r except 0 and 15. Final weights 
are listed in Table 4 according to number of comparisons. (Only 
some of the weights obtained displayed.) Since n=7, the number 


of comparisons possible ranges from 21 to 6. 
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PERSONNEL EXAMPLE USING ADDITIVE MODEL WITH 7 CRITERIA 

+m AND 7 APPLICANTS IMPACT ON FINAL WEIGHT VECTOR 

= OF DECREASING NUMBERS OF COMPARISONS 

NO 

* 

“. 

n=7 Absolute Deviation 

ns 

& Comparisons, k 
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This demonstrates that in a real-life decision making scenario, 
the absolute deviation technique can produce a believable result 
even when many comparisons are omitted. The decision maker who 
generated the set of final weights from 21 comparisons in each 
matrix was complemented by the rest of the search committee for 
the impressively thorough nature of his decision report. They 
agreed with the results of his ranking. 

3: CONSISTENCY AND RELIABILITY 

The additive model matrices generated in the Personnel 

Example were considered individually and compared to provide 
further data on the relationship of the number of projects 
compared, n, with the number of comparisons, k, (ranging from 
M = (n)(n-1)/2 to (n-1) and the objective function, 2, for 
matrices A - H of differing consistencies. Figure 1 shows the 
average value of 2 for each number of comparisons for n=7. If 
the observations from each matrix A - H were fit to a straight 
line, then for a matrix, say A, the slope ma of line La is the 
ratio of the change in objective function AZ to the change in 
number of observations, A k. If it is assumed that this slope 
is constant for all k, it provides a measure of reliability based 


on the objective function and number of comparisons. 
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Since the scale, s, (range of input values) influences the 
maximum inconsistency, a reliability index based on 2/k should be 


normalized with respect to scale. If 


z= (s-1) (arb (30) 
M 2 


is taken as an estimate of the worst error, Zu! for a matrix of 


size n generated with scale s, then a possible reliability index 
: RI, is 
RI 22 


: ee ee (31) 
k(S-1)(n-)V) 


statistical evaluation of the above result and investigation of 

the robustness of the assumptions is ongoing. Future experiments 

should investigate the linearity of Z/k, the impact of n, and the 

effect of s on the proposed reliability index. 

4, EXAMPLES 

. Further examples of the use of the Basic Model 1, Additive 
Model 2, Voting Model 3, Hierarchy Model 4, and the Constraints 
Model 5, are found in Appendix A, the User Manual. A composite 
example incorporating all the models is reported in Section VI of 
the User Manual, and Appendix B, Additional Examples, contains 


detailed input and output information on other applications of 


y the system tested. Appendix C contains a copy of the commented 
: computer program implementing the methodology described herein. 
@ 
3 
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SECTION V 


CONCLUSIONS 


1. SUMMARY OF RESULTS 

A feasible and instructive method has been developed for 
ranking projects or making other decisions such as personnel 
selection, policy making, presentation and allocation of merit 
awards. The structured methodology now implemented on "VAX" and 
"IBM" computers provides insight into the decision making process 
for the decision maker and everyone to be apprised of the 
decisions. In addition to facilitating communication, it 
establishes credibility and is a powerful tool in investigation 
of the processes being considered, and in negotiations. 

The currently implemented program provides a mechanism for 
selection among up: to 200 projects based on their overall worth 
or on up to 20 different criteria. Voluminous input required by 
other systems is reduced by allowing many preference comparisons 
to be omitted. Consistency increases when fewer observations are 
input since the user is not forced to guess when aeereean of 
some comparisons. The reliability of the final weight vector 
increases with the consistency of the input and with the 
percentage of possible comparisons actually input. Further 
javestieaeien is proposed eaguEaiag the effect on the consistency 
and reliability indices suggested of the number of items 
compared, the comparison scale employed, and the number of 


missing observations. More applications of the system are 
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planned and simulations could be designed to table the 
consistency and reliability indices. Future enhancements to the 
computerized system will be based on feedback from the defense 
department laboratory currently using the system. 

2s RECOMMENDATIONS 

A useful avenue for further research would be to more fully 

implement the existing research and analytic programs. Such a 

: system should include (1) Streamlined programs (2) Data input 
(3) Storage (4) Matrix Generation (5) Merging (6) Error Detection 
(7) Training (8) Use of other data bases and (9) Stand-alone 
Programs for the 2-100. 

The first step would be to collect the opinions of the users 
to make the basic programs more responsive to needs of the 
various users. One important task would be to develop a 
stand-alone version of the program that could be executed at the 
user's desk or taken to another location. Methodology might be 
altered so that, by using least squares procedures, preference 
values in the comparison matrix could be estimated prior to going 
into the weighting process. Other features of new research would 
be to improve the procedure for determining which comparisons are 
contributing heavily to the inconsistency of the result. Also, 
sensitivity. analysis would be.made available to determine the 
impact of errors on the final outcome. The effect of scale would 
be investigated further, and the hierarchy model refined. 

More specifically, a follow-on study would entail 
development of an integrated MIS system based on the 2-100 as a 
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stand-alone computer. Where the 2-100 size limitations constrain 
the model, an interface with the "VAX" computer and currently 
existing programs could be provided. Data-base information could 
also be retrievable from the "CYBER." The Basic, Additive and 
simple Voting models and a tailored version of the Hierarchy 
model would be available using the 2-100 as a stand-alone 
computer. Accessing of the models would be aided by an 
interactive front-end program on the 2-100 which could also 
direct any links with the “VAX" or "CYBER" needed to access 
data~base records of projects, run 2 sensitivity analysis, 
provide optional access to stored files, do extremely large 
examples, use an improved voting model such as the one described 
below, or execute an enhanced Constraints model. Development of 
the interface would be enhanced by working closely with potential 
users, receiving the assistance of one designated person at the 
laboratory, and helping potential users to write personalized 
operating instructions. 

Features of the 2-100 based integrated Management 
Information System would, for example, allow the decision maker to 

* enter data and execute tailored versions of the Basic, 
Additive, Voting and Hierarchy models, 
experiment with several runs of various models while 
remaining at his own work station and protecting his 
privacy. 
avoid any (highly unlikely) breaches of security due 
to unauthorized access of the "VAX," 
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v * employ interfaces with the the "VAX" and "CYBER" for 
- optional file storage and access to group member files, 
- executing runs with large numbers of comparisons, 
- running a sensitivity analysis, 
- executing a newly proposed Voting model, 
- executing the Constraints model, and 
- interfacing with task description and accounting 
records. 

The implementation of the system on the Z-100 could involve 
breaking the existing software into modules and reducing the 
number of possible comparisons entered at one time. A new model 
for estimating ranks and weights could free the 2-100 porticn of 
the system from the licensed software package, "LINDO", thus 


enhancing portability. The new model estimates weights as 


@ = (BiB)! ple (32) 


where W is the estimated weights vector and the B matrix is 
derived from the input comparisons matrix A. Matrix B.has one 
row for each of the C non-zero Sompardsons in the matrix, A. For 
each comparison ayy’ B has a row, c, with only 2 nonzero 
elements: 


b. = 71 (33) 


and 
DA ae (34) 
Row C + 1 of matrix B is all ones. 
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“ This least squares procedure provides a reasonable estimate 
of ranks and weights, but in the rare event that the matrix B is 
singular, the user Would have to use the "VAX" system. 
Singularity is not a problem with "LINDO." The actual number of 
comparisons which could be accommodated on the Z~100 would be 
determined during further studies. 

Further study would thus provide additional advantages such 

as: ‘ 


- facilitating user access by implementing portions of the 
system on the 2-100. 


€ 


~ linking with task description and accounting databases to 


provide the decision maker ready access to such data as the 
description of a task and its projected cost over the next 5 
years, money spent, and money remaining unconstrained, 

- improving the Voting model to provide a better estimate 
when small groups are voting on a small number of tasks by 
testing and refining a new model. After each of the V voters 
ranked the same n projects and was satisfied with his 
individually produced weights Wig! Woye Way! the Re comparisons, 


a , Of each voter, vy, would be combined to produce final 


ijv 
weights w 


1’ wees Perhaps using the model: 


Vv K + - 

min : = (€ ££ )=2 (35) 
v=l k =l vk vk 
Vv Vv Vv 
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for v=l...V: k =1...K 7; 1 
Vv Vv 


n-l < K <n (n-1)/2 


An important consideration in the system would be to 
determine what programs should be ranked. Any set of programs 
may be weighted, but for reasons of efficiency and time, certain 
programs may not be considered. For example, all 6.2 work ought 
to be weighted, but the time involved would be greater. It might 
be better to combine work units into work packages for ranking at 
the division and lab levels. Further, 6.2 lab funded programs 
may be desirable to breakout. Still further, micromanaged, 
mortgaged, and very important programs may be removed. The . 
remaining set would be more manageable and would include only 
those programs about which there is some doubt about funding. 


The work here would develop a screening method from interviews 


with key personnel to insure that only those requiring a decision 


are ranked. 
A further consideration would be to implement at higher 


levels of the organization existing programs which reduce the 
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effort involved in ranking. The members might thus be able to 
rank the laboratory programs uSing as few as three comparisons 
even though such results would not be as strong as those produced 
from more comparisons. 

Future efforts could include designing a system for 
coordinating the rank program with the existing financial 
database and storing the information deneparea by the ranking 
process. The input and output formats of the system would be 
specified. Personnel in DLX would be asked to participate in 
this specification by communicating their needs and preferences. 
Methods for implementing the system would be investigated. DLX 
would provide instruction with regard to the current financial 
database and how it is accessed and updated. If the financial 
database could easily be automatically linked to the ranking 
programs, this would be done. Otherwise, DLX would provide a 
manual link or the link would be constructed in a future effort. 
If improvements in the current data base and procedures were 
needed, recommendations would be submitted. A suitable language 
for writing the system program would be determined. The program 
would be written and documented, and trial runs would be made 
using DLX personnel input. 

The Seal thrust effort would be summarized after the final 
selections were made to determine if the lab program is 
emphasizing the currently identified needs for certain 


capabilities. 
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Verification and validation should be done before any system 
could be viewed as complete. The entire system would be 
verified by going over all the options in the training sessions 
and determining whether the program produces the anticipated 
results and how closely those results conform to the overall 
feelings of the user. This step would be time-consuming, but 
existing test examples indicate that a verifiable program could 
be produced with effort. 

The validation of the programs would be more difficult. By 
validation it is meant that the high ranked projects do tend to 
be the high payoff projects in the end. The present program 
provides only for setting up validation procedures using the 
audit trail allowed by the system. The reason for this is that 
most programs to be ranked go for 2 or more years, but this 
effort would be completed in less than 2 years. The procedures 
necessary for completion of the validation would be included in 
the final report. 

After the system was programmed and all the procedures: 
including optimization were worked out to the satisfaction of key 
personnel, the users of the system would be trained. For this 
purpose, one person in DLX should be designated to learn the 
Operation of the programs thoroughly. Also, the investigators 
would help users write operating instructions and train all users 
the laboratory designated and made available, not to exceed one 
from each branch, two from each division, and six others from the 
laboratory. 
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In addition, the system program would be maintained by KR. 
Further efforts would include the training of KR personnel on the 
program in addition to providing the instructions and the 
documented program. 

Investigations would be carried out concerning the 
possibility of making: 1. cut or delay decisions, 2.. group 
decisions, 3. extending this methodology to SENSE laboratories, 
and 4. determining manpower requirements using top down - bottom 
up planning procedures. . 

If budget restraints force a prGorsi to be discontinued 
during the current or coming year, the question arises as to 
whether the program effort should be stopped or simply delayed. 
In order to solve this problem by a model, rankings for FY + 2 
are necessary. The investigation of this problem would consider 
the feasibility of making realistic rankings 2 or more years 
ahead. If such rankings were feasible, the effort and cost 
involved would still need to be studied to determine if the 


potential value is great enough to justify the cost. 
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APPENDIX A 
USER MANUAL: RANKING SYSTEM FOR PROJECT SELECTION 


1. DESCRIPTION 

Rank and weight projects on an individual basis with regard to oerall 
worth or selected attributes. Vote on project ranks as an equal group; or, as 
part of the hierarchy, retrieve subordinates' rankings, modify and/or 
supplement them. Maximize the number of projects funded based on their 
individual costs, final weights, and overall budget. Provide an audit trail 
of the decision making process, if desired. 


¢ 


2. CONTACT 


Dr. Charles R. White 
Industrial Engineering Department 
Dunstan Hall 
Auburn University, Al 36849 
(205) 826-4340 


January, 1986 
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a. OVERVIEW 


Goal - Provide a decision support system to improve the project selection 
mechanism and provide an optional audit trail. 


Method - Use one or a combination of the five models to configure a decision 
support system for your particular application. 


BASIC Model 1 - Rank projects based_on overall worth 


Input: List of projects and comparisons based on overall worth. 
(Program estimates missing comparisons. ) 


Output: Project ranks and weights and an estimate of the consistency 
* of comparisons. 


ADDITIVE (Attributes) Model 2 - Rank projects based on multiple criteria. 


Input: List of projects and attributes, attribute comparisons, then 
project comparisons with respect to each attribute. 
(Program estimates missing comparisons. ) 


Output: Attribute ranks, weights and estimate of consistency of 
comparison; within each attribute, project ranks, weights and 
consistency estimates; then, final ranks and weights of projects. 


VOTING Model 3 = Combine a committee's project weights. 


Input: Project weights produced by execution of other models (using 
the same set of projects). 


Output: Final project ranks and weights. 


HIERARCHY Model 4 - Modify and combine subordinates' project weights. 


Input: Project weights produced by execution of other models by 
subordinates ranking different projects; list of projects and 
comparisons to be added, modified or deleted; new comparisons 
connecting the (revised) subordinates’ comparisons (program estimates 
missing comparisons). 


Output: Final project ranks and weights and consistency estimate. 


CONSTRAINTS Model 5 - Allocate scarce resources, such as money. 


Input: List of limiting constraints (such as costs, manpower, 
facilities); project weights produced by executing other models; for 
each project and each constraint, the requirements of the project. 


Output: List of projects to fund and value of the objective 
function. 
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b. DIRECTIONS FOR USE 


(1) STEP 1: Select Model(s) and Prepare Input 


(a) BASIC Model 1 Rank Projects Based on Overall Worth. 
The weights produced by executing this model may be final or they may 
be used again. At a later time, comparisons may be retrieved from a file, 


viewed and changed before rerunning Model 1. Alternatively, the BASIC model 
can be run as a prerequisite to the VOTING, HIERARCHY, or CONSTRAINTS Model(s). 


1) New Set _of Camparisons 


Using an input form copied from Section III of this manual, fill in 
information about the projects to be ranked. 


EXAMPLE OF LIST STYLE INPUT 


NAME: _padgettm INPUT FILE NAME(S)(if any) 


ORGANI ZATION : 


DLXB OUTPUT FILE NAME dilxbla.dat 


PURPOSE: compare projects NAME OF ATTRIBUTE (if any)overall worth 


NAME (LIST 
NOT 
NUMBER ATTRIBUTES } PREFERRED PREFERRED 


TIE WEAK STRONG ABSOLUTE 
123 4 5 6 7 8 9 


WWWYNY WU Ww 
PU PRU BMH REE 


First, list projects then perform pairwise comparisons of projects. Beginning 
with the first project, compare it with a project below it on the list. For 
example, to show you prefer PR2 over PR1 by a degree of 5 (strongly): write 2 
in the first row under PREFERRED, write 1 under NOT PREFERRED, and put a5 in 
column 5 of the same row. For each project on the list, compare it with at 


least one project below it. The program will estimate missing comparisons. 


See Model 1 system flow diagram in Section A-3.d.(1) and execution of Model 1 
using the above input in Section A-3.e.(1). 
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EXAMPLE OF MATRIX STYLE INPUT 


(Optional input style especially desirable for HIERARCHY Model 4) 


NAME Input file name (if any): 


ORGANI ZATION : Name of output file to contain results: 
(See rules for file name selection below. ) 


PURPOSE: Name of attribute (if any):_ 


PROJECT OR PROJECT (OR ATTRIBUTE) COMPARISONS (see rules below) 
ATTRIBUTE NAME ~ 
FOR FILE (Up to 32 spaces long) 


Up to 200 total projects and up to 20 attributes may be used. 


Rules for camparisons: 


Choose a scale: x to y where 0 < x <y < 50 and x and y are real numbers. 
(Each user may choose own scale) 
For selected pairs, enter comparison. 


For example, if you choose a scale of 1 to 9 and prefer PR2 over PRI by a 
degree of 5 out of 9, enter 5 in row 2, column 1 of matrix. If you had 
preferred PRl over PR2, a number should have been entered in row 1, colum 2 - 
diagonally opposite to the 5 you entered. If PR3 is preferred over PR4 by a 
factor of 8 out of 9, enter 8 in row 3, column 2. I£ you do not prefer either 
project, indicate a tie by entering al in either row 2, column l or rw l, 
column 2. : 


Missing comparisons will be estimated by the program. User must input 
enough comparisons to connect the rows by being sure to campare each listed 
project with at least one project below it on the list. A comparison in row 
i, column j automatically supplies a value for row j, coiumn i. All diagonal 
values are set to l by the program. 


Hf 
Ms 


Figure A~l. Example of File Name Selection 


Suggested rules for file name selection: 


The left most letters of the filename are the user's organization letters. 
The second from right digit is the number of the individual user (1-5 at each 
level). The rightmost letter (a to z) indicates the particular file. A 
user's official file must have a rightmost zero (0) instead of a letter. All 
user accounts must be part of the LINDO group on the VAX 2. The system will 
automatically prefix the user's last name to the file name. The file type 
should be DAT. 
Examples: [PADGETIM) DLXBla.DAT 
[PADGETTM] DLXBIO.DAT (official) 
2) Recheck of earlier run of BASIC Model l. 

Print off old output file (e.g., TYPE filename). 


Prepare corrected input using one of the input forms described above. 
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(b) ADDITIVE (Attributes) Model 2 


1) New weightings 
LIST OF INPUT TO PREPARE FOR ADDITIVE MODEL: 


a) Prepare input for 1 to 20 attributes as for BASIC Model 1, listing names 
and comparisons for each attribute. 


b) For each attribute, prepare input as for BASIC Model i, using up to 200 
projects with respect to that attribute. 


This results in one form for comparing attributes and separate forms for 
comparing projects with respect to each attribute. All will use the same 
output file. See the system flow diagram for Model 2 and the example of Model 
2 in Sections A-3.d.(2) and A-3.e.(2), respectively. 


2) Repeated weightirigs 


Prepare input as described above, indicating changes to be made in 
comparison values. List names of input files where current information is 
stored. 


(c) VOTING Model 3 


All voters have equal weight and each must have previously weighted the 
same projects. 


EXAMPLE OF INPUT VOTING MODEL 3 


Number of voters (2-5): 2 

Number of projects: 3 

Name of official input file for each voter: 
{PADGETIM] DLXB10.DAT 

{SMITH] DLXB20.DAT 


Project Project 
Number in Name in 
All Files All Files 
a 


ae PRL 
2 PR2 
3 PR3 


Note: If an inconsistency in the number of projects in each file is found, 
the lowest number will be used. If an inconsistency in the names of projects 
is found, the names from the first file will be used. Any inconsistencies 
will be reported to the user. Only official files (ending in 0.DAT) should be 
used for input! See examples in Sections A~3.d.(3) and A-3.e.(3) and A-3.f. 
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(d) HIERARCHY Model 4 


First, have subordinates produce individual lists of weights using BASIC 
Model 1 or ADDITIVE Model 2, storing the results in an official file. If 
individuals are ranking the same tasks, results should be pooled using VOTING 
Model 3 (and/or HIERARCHY Model 4) to produce group lists of weights. Up to 20 
lists, each ranking different projects, may be input to the HIERARCHY Model 4. 


Second, retrieve each list of ranks and weights (e.g. using TYPE 
filename). Approve and/or modify these weights. (Remember the actual 
comparison matrices are stored in official files produced by Models 1, 2, 3, 
or 4). Omit projects certain to be funded or dropped. Add new projects if 
desired. The new matrix could be viewed to confirm agreement with earlier 
weights or spotlight needed changes. 


Third, using a list of projects for each individual or group, fill in an 
input form as illustrated below. (Program will output this list for you. 
Input forms can be copied from the one inthis user guide.) 


¢ 


EXAMPLE OF INPUT FOR HIERARCHY MODEL 4 


Project Name 

& Number in 

Ascending Camparisons to be 

Order as in changed or added 
Source Input File 2 3 4 5 6 


Subl:DLXB10 //fA 11 
//. 


//, 
Subl : DLXB20 2) //A /S/A 11 
3} //A /// /9 
FLA: ASS. 
New projects 5 
added by 
supervisor 2 
(underlined) 
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Note: Shaded areas represent matrices constructed by program from input 
files. Supervisor may still change entries here. Supervisor should add 
enough comparisons to connect all rows. Program will estimate missing values. 
See instructions for BASIC Model l input for more information on file name 
conventions and both list and matrix style input forms. 


System flow diagrams and HIERARCHY Model examples are found in Sections 
A-3.d.(4) and A-3.e.(4), respectively. One possible configuration using all 
the models is illustrated in Section A-3.f. 


Temporary diagonal convention: In order to override subordinates' 
preferences appearing below the diagonal of the subordinate's, individual 
preference matrix, input the new preference as j over i by degree k and zero 
out the inverse of the original preference by inputting a new preference of i 

. over j by degree 0. For example, to change new project number 5 to prefer 5 
over 4 by degree 4 enter a 4 in row 5 column 4 and also enter a zero in row 4 
column 5. For new (underlined) comparisons linking subordinates' matrices, 
only one entry is necessary. . 


¢ 
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(e) CONSTRAINTS Model 5 


First, choose one or more constraints such as costs, manpower, and 
available facilities upon which to base the funding decision. Next, a unit, 
such as dollars, man hours, or hours available is input. Third, the total 
resources available for each restraint must be indicated. Fourth, resource 


requirements for each project are determined and entered using a list similar 
to the one below. 


INPUT FOR CONSTRAINTS MODEL 5 


UNITS TOTAL RESOURCES * 


Thousands of Dollars 5,000 
Man Months z 100 


RESOURCE REQUIREMENTS FOR CONSTRAINTS 


System flow charts all indicate that the CONSTRAINTS Model may be entered 
following any other model(s). See Sections A-3.d.(5) and A-3.e.(5) for a 
system flow diagram and example of model execution. The composite example in 
section A-3.f. also uses the CONSTRAINTS Model. 
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(2) STEP 2: LOGON to Eglin VAX 2 Computer and Enter LINDO 
System 


The Eglin VAX 2 can be accessed from a laboratory terminal or from a terminal 
using a dial-up modem. 


(a) Dial-up Using model (optional) 
(For LOGON from laboratory omit this section. ) 


Your dial-up terminal should be set to emulate a VT-100. Configuration 
required includes 8 bit, no parity, 1 stop bit, and no local echo. 


Phone lines for 300 Baud or 1200 Baud are available. If one number is 
busy, dial the other one. Phone numbers are subject to change. 


DIAL-UP PHONE NUMBERS 


¢ 


300 Baud Lines 


(904) 882-8691 
-8692 


1200 Baud Lines 
(904) 882-3811 


-3798 
-8329 


A, 

wa 

x4 

eS (b) LOGON from laboratory terminal 

; Logon to Eglin VAX2 camputer and enter LINDO system. (Caps represent system 

output; underlined small letters are user input; remarks are in parentheses.) 

4 EXAMPLE OF LOGON AND ENTRY TO LINDO SYSTEM FROM LABORATORY TERMINAL: 

8 SYSTEM: UV2 (return) 

‘a. 
USERNAME: Colburn (return) 

= PASSWORD: 4AC23A8 — (return) : 

~ $: Lindo (return) : . 

ai 2 Gases iretuen) (Specify model and files) . 

= (3) STEP 3: Executive Desired Model and Interpret Results 

Respond to system prompts. Use prepared input. Refine your weights by 

Ae repeating as needed. See system flow diagrams and examples in Sections 

len A-3.d., A-3.e., and Anv3.f. 

Interpretation of results can be quick and simple or intensive and recursive, 

me according to the user's need and his satisfaction with preliminary results. 


Briefly, the output from BASIC Model 1 consists of a ranked and weighted 
list of the ,rojects (or other items) compared. The objective function (OBJ) 
represents the sum of the errors due to inconsistent input by the user. The 
‘. index is the objective function divided by the mumber (n) of projects 
ne compared. The optionally viewed Delta matrix displays the difference between 
ae the user's input, a.., and Wi/W 3, the input expected for row i and column } 

for perfect consistércy. So, a large number in row i column j of the Delta 
Matrix, say diye gives a possible source of inconsistency. If there is a high 
inconsistency, and dj j is large the user should reevaluate his comparisons, 
particularly those involving projects i and j. For more detailed instruction, 
the user may refer to the body of the final report and/or contact the 
principal investigator. 


(4) STEP 4: LOGOUT, Save or Delete Output File 
ENTER "QUIT" TO LEAVE LINDO 


“s : quit (return) 
; SAVE OR DELETE OUTPUT FILE(S/D) 5 (return) 


ENTER "LOGOUT" TO LEAVE VAX2 
: logout (return) 


ae sey! 
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on INPUT FORMS 


Example input forms for list, matrix, and constraints model may te duplicated 
for your use in planning program input. 
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d. SYSTEM FLOW DIAGRAMS 


The following diagrams outline the user input needed for each model type 
and indicate possible responses to the output from each model. The different 
models may be treated as modules forming a complex system, or run as 
stand-alone additions, or updates to the database. 
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Figure A-2. (1) Basic Model 1 (with refinements and constraints .:tional). -} 
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Figure A-3. (2) Additive (Attributes) Model 2. 
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Figure A-6, (5) Constraints Model 5. 
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APPENDIX B 


ADDITIONAL EXAMPLES ’ 


1. TEST EXAMPLE 


Results from ranking projects using the Saaty~based program developed 
during Phase I [3] to the results from using the artificial deviation 
technique with a standard linear programming package to illustrate the 
effectiveness of the new technique even with missing comparisons when input 
is quite consistent. Eliminating a constraint is equivalent to amitting a 
comparison. The computer input and output below shows that a non-technical 
user needs an interactive frontend such as the one provided by this 
project. The objective function values and the weights given appear in 
Table B-l. 
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_ el elt el Oe Pee! ‘- 3 ) de Be) an ee eg rs RE a ae ae ag ae ig ge wg re 


Model 7 BASIC 
SAMPLE PROBLEM 


1.00 0.20 0.11 0.50 0.33 


: 5.00 1,00 0.33 3.00 2.00 
wf 9.00 3.00 1.00 8.00 4.00 
~ 2.00 0.33 0.13 1.00 0.50 ; 
“ 3.00 0.50 0.25 2.00 1.00 


SETUP 


MIN Sl +S2 +S3 +S4 +S5 +S6 +S7 +S8 +S9 +S10 +S1ll +812 + $13 


: + $14 +515 + S16 +S17 +S18 +519 + $20 
> SUBJECT TO 
5 (1) -Swl + we - sl+s2 =0 
™ (2) -9wl +3 - s3+S4 =0 
(3) -2wl + WA - $5+S6 =0 
2 (4) -3wl + W5 - $7 +S8 =0 
, (5) -3W2 + W3 - s9 +510 =0 
= (6) - w2 +3W4 - $ll +s$12 =0 
. (7) - w2 +2W5 -$l13 +Sl4 =0 
. (8) -wW3 +8w4 -s15 +S16 =0 ; 
. (9) - W3 +405 =§17 +518 = 0 
: (10) -2w4 + WS - $19 +520 =0 
¥ (11) Wl + W2 + W3 + W4 + WS = 1 
- 
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ABSOLUTE DEVIATION 


0.045455 
0.204545 
0.545455 
0.068182 
0.136364 
0.318181 


REMOVE SOME OF THE CONSTRAINTS 


REMOVE 
CONSTRAINT # 


1 


Ea a a a OY 


= 


eo 


5 


al | 


rh SE 


Zz. SAATY SCHOOL EXAMPLES 


a. Output Files Documenting Table 2 
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a a a a I IT IL SL RY EIST T  e 


00000000 1 00000000 0 00000000 oO 00000000 O 00000000 O 00000000 oO 
00000000 0 00000000 1 O0000000 0 O0000000 0 00000000 0 00000000 0 
00000000 0 00000000 ‘0 00000000 1 00000000 0 00000000 0 00000000 0 
00000000 0 00000000 ‘0 00000000 ‘0 00000000 ‘1 00000000 0 00000000 oO ae 
00000000 0 00000000 0 00000000 ‘0 00000000 0 00000000 1 00000000 0 ie 
00000000 00000000 & 00000000 ‘1 00000000 € NOO00000 00000000 1 

S3INWA ONISSIW N31 ‘3SV) INBLISISNOINI <t oo FiGeTBlLyY YOd XIMLYW LDBCOudTI = 


00000000 | 00000000 0 00000000 £& 00000000 9 00000000 0 OC000000 0 
00000000 0 00000000 1 00000000 oO 00000000 ¢ 00000000 ¢ 00000000 0 
00000000 O 00000000 0 00000000 ft 00000000 0 00000000 0 ©0000000 0 
00009000 ‘0 00000000 0 00000000 0 00000000 | 00000000 0 00000000 0 
00000000 0 00000000 0 00000000 € 00000000 0 00000000 1 00000000 0 
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